Papers by Tim G. J. Rudner
SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for uncertainty quantification in large language models provide little insight into factors responsible for an uncertainty estimate, limiting their usefulness as practical tools for improving trustworthiness and understanding uncertainty reasoning. |
| Approach: | They adapt causal tracing and zero-ablation techniques to study the effect of different circuits on LLM generation to identify whether factuality of generated responses and uncertainty originate in separate or shared circuits. |
| Outcome: | The proposed methods use the well-established methods of causal tracing and zero-ablation to study the effect of different circuits on LLM generation. |
Simple Factuality Probes Detect Hallucinations in Long-Form Natural Language Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to detect hallucination require many samples from the LLM generator . current methods require multiple samples, which is computationally infeasible . |
| Approach: | They propose a simple baseline for detecting hallucinations in long-form LLM generations . they show that LLM hidden states are highly predictive of factuality in long form natural language generation . |
| Outcome: | The proposed method is comparable to expensive multi-sample approaches while drawing only a single sample from the LLM generator. |
MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for faithful calibration of large language models (LLMs) are insufficient and can harm faithful calibration. |
| Approach: | They propose a new prompt-based calibration approach inspired by human metacognition that measures faithfulness across diverse models and task domains and enables up to 61% improvement in faithfulness. |
| Outcome: | The proposed approach improves faithfulness across diverse models and task domains and achieves an 83% win rate over original generations as judged by humans. |